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® Method and apparatus for learning of neural network. 



0 An apparatus for carrying out learning operation of a neural network which has an Input layer, an 
Intermediate layer and an output layer. Plural nodes of the Input layer are related to plural nodes of the 

a intermediate layer with plural connection weights, while the plural nodes of the intermediate layer are also 
related to plural nodes of the output layer with plural connection weights. Although input data composed of plural 
Q> components and teaching data composed of plural components are used in learning operation, some of the 
components are ineffective for the purpose of learning operation. During error calculation between output data 
from the neural network and the teaching data, the apparatus judges whether each of the components of the 
00 teaching data is effective or Ineffective, and output errors corresponding to the Ineffective components are 
JN regarded as zero. The connection weights are thereafter corrected based upon the calculated errors. The 
apparatus further comprises means for adding new Input data and teaching data into a data base, and means for 
O calculating a degree of heterogeneousness of the new teaching data. The new input data and teaching data are 
added to the data base only when the degree of heterogeneousness Is smaller than a predetermined value. 
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BACKGROUND OF THE INVENTION 
Field of the Invention: 

5 The present invention relates to a method and an apparatus for learning operation of a neural network. 
Discussion of the Prior Art 

A neural network has been known as a network which memorizes a sequence of cause and effect and 
10 which is optimized by changing connection weights thereof through teaming operation. For example, such 
neural network is composed of an input layer, an intermediate layer and an output layer, and each of layers 
having plural nodes. Each of the nodes of the input layers is related to the respective nodes of the 
intermediate layers with respective connection weights or connection strengths, while each of the nodes of 
the intermediate layers is related to the respective nodes of the output layers with respective connection 
is weights. During learning operation, dispersed plural input data are given to the neural network so as to 
correct or modify the connection weights so that reasonable output data are always obtained when any 
input data are given. 

Namely, one pair of input data and teaching data are firstly used to correct the connection weights so 
that the output data from the neural network approach to the teaching data, which represent optimum values 
20 corresponding to the input data. This learning operation is repeated using a different pair of input and 
teaching data, and the teaching operation is stopped when all the pairs of input and teaching data are used 
for the learning operation. After the learning operation, the neural network exhibits a desirable input-output 
characteristic. 

The above-mentioned neural network is recently used in machine tools for calculating optimum 
26 machining data based upon input and fixed condition data so as to properly carry out required machining 
operations. 

In a neural network system used in the machine tools, each of the input data is composed of plural 
components or plural data items, such as material of a workpiece to be machined, surface roughness, 
dimensional accuracy and the like, while each of the output data is composed of plural components or 
30 plural data items representing machining conditions such as rotational speed of the workpiece, infeed speed 
of a tool and the like. 

However, all of the components of the input data and all of the components of the output data are not 
necessarily used all the time. There is a case in which particular component or data item is used in a 
particular pair of input and teaching data, but not used in other pairs of input and teaching data. 

as For example, in grinding machines, several types of grinding cycles are carried out, and each of the 
grinding cycles uses different data items as machining data. In a pair of input and teaching data for the 
cylindrical grinding, data for shoulder grinding and corner grinding are omitted. In a pair of input and 
teaching data for shoulder grinding, data for corner grinding are omitted. Even in such event, the number of 
the input nodes of the neural network, namely the number of the nodes in the input layer of the neural 

40 network is set to be equal to the number of all the input data items or components of the input data, white 
the number of the output nodes of the neural network, namely the number of the nodes in the output layer 
of the neural network is set to be equal to the number of all of the output data items or components of the 
output data. In such system, predetermined constant values are given to the input nodes as data for omitted 
input data items, while some components output from the output nodes corresponding to the omitted data 

45 items are ignored, because the ignored components take meaningless values. 

Such system therefore has a problem in that the connection weights of the neural network are corrected 
based upon the meaningless values of the ignored components, whereby the input-output characteristic of 
the neural network is modified in a wrong direction. This problem becomes more serious when the number 
of the omitted data items increases. 

so Further, a problem sometime occurs in the conventional neural network system in that output date from 
the neural network take inappropriate values. 

In such case, the input-output characteristic of the neural network must be corrected, and a pair of new 
input data and new teaching data are further added to the data base to carry out a renewal teaming and 
thereby to correct the input-output characteristic thereof. 

55 However, the addition of the new input data and new teaching data may sometime cause a drastic 
change of the input-output characteristic, whereby a part of the input-output characteristic which is to be 
maintained is also drastically changed. 
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Therefore, an operator experimentally judges whether or not the new input data and new teaching data 
can be used, namely, whether or not these data match with the tendency of the past learning process of the 
neural network. The operator then adds the new input data and new teaching data into the data base for the 
renewal learning of the neural network, only when the new input data and new teaching data match with the 

s tendency of the past learning process. 

As described above, in the conventional system, new input data and new teaching data must be 
evaluated by the operator with his experiment and sixth sense. Therefore, the renewal leaning operation has 
been time consuming and very difficult work. Further, the neural network may sometime be renewed 
undesirably, whereby the input-output characteristic of the neural network is inappropriately changed to 

10 have an undesirable characteristic. 

SUMMARY OF THE INVENTION 

Accordingly, it is an object of the present invention to provide improved method and apparatus for 
js property carrying out the learning operation of a neural network. 

Another object of the present invention is to provide improved method and apparatus for properly 
carrying out the learning operation of a neural network even in cases where some data items are omitted 
from some of input data and teaching data. 

Still another object of the present invention is to provide improved method and apparatus for carrying 
20 out the learning operation of a neural network wherein a renewal learning operation is carried out to conform 
with the tendency of the past learning process of the neural network. 

Briefly, a neural network according to the present invention has at least an input layer comprising plural 
input nodes and an output layer comprising plural output nodes. During the learning operation of the neural 
network, plural pairs of input data and teaching data are used. Input data of each pair given to the neural 
25 network comprises plural components to be applied to the input nodes of the neural network, respectively. 
The teaching data of each pair also comprises plural components corresponding to the respective output 
nodes of the neural network, and represents optimum or desired output values to be output from the neural 
network when corresponding input data are input to the neural network. 

According to the first aspect of the present invention, ordinal error calculation is carried out for effective 
30 components, but special treatment is taken for ineffective components. Regarding effective components, 
output errors are calculated using the effective components, namely the output errors are calculated based 
upon differences between the values of the effective components and corresponding output values from the 
neural network. On the contrary, regarding the ineffective components, the output errors are regarded as 
zero. Namely, output values from the output nodes corresponding to the ineffective components of the 
35 teaching data are treated as correct values, and the connection weights of the neural network are corrected 
under this assumption. 

With this arrangement, it is possible to prevent the ineffective components from affecting the connection 
weights of the neural network. Accordingly, even if a learning operation is carried out using input data and 
output data comprising ineffective components, the characteristic obtained through the past learning 

40 operations which have been carried out using other teaching data are not affected by the values of the 
ineffective components. As a result, it is possible to properly carry out the learning operation of the neural 
network even in cases where some data items are omitted from some input data and teaching data. 

According to another aspect of the present invention, differences between output data and teaching 
data are calculated, as a degree of heterogeneousness (hereinafter referred to as "heterogeneousness 

45 degree"), every time when new input data and new teaching data are given for the renewal learning of the 
neural network. The heterogeneousness degree is compared with a predetermined threshold value. When 
the heterogeneousness degree is smaller than the value, the new input data and new teaching data are 
added into the data base for learning operation. When the heterogeneousness degree is larger than the 
value, the new input data and new teaching data are not added into the data base. 

so With this arrangement, it is possible to prevent heterogeneous input and teaching data from being 
added to the data base. Therefore, it is possible to prevent the neural network from having an input-output 
characteristic outside of an agreeable range. 

Further, in a preferred embodiment, the threshold value is changed in accordance with the number of 
the renewal learning. In this case, the fitness of the new input data and new teaching data is judged taking 

55 the progress of the learning operation into consideration. 

According to the second aspect of the present invention, it is possible to efficiently carry out optimum 
learning operations without relying on operator's experience and sixth sense. 
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BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS 



Various other objects, features and many of the attendant advantages of the present invention will be 
readily appreciated as the same becomes better understood by reference to the following detailed 
5 description of the preferred embodiment when considered in connection with the accompanying drawings, 
in which: 

FIG. 1 is a block diagram showing a computer system having a neuraJ network according to a present 
invention; 

FIG. 2 is a chart showing the structural model of the neural network; 
10 FIG. 3 is a flowchart showing a processing of the CPU shown in FIG, 1 which realizes the operation of 
the neuraJ network; 

FIGS. 4 (a) and 4 (b) are flowcharts showing a processing of the CPU during a learning operation; 
FIG. 5 is a chart showing the contents of a data base; 

FIG. 6 is a chart showing a discrimination data table for memorizing data indicating whether each 
is component of teaching data is effective component or ineffective component; 
FIG. 7 is an explanation chart showing data items of the input data; 
FIG. 8 is an explanation chart showing data items of the teaching data; 

FIG. 9 is a flowchart showing a processing of the CPU for adding new input data and new teaching data 
in to the data base; and 

20 FIG. 10 is a chart showing a relationship between the number of renewal learning operations and a 
threshold value. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
25 Referring now to the drawings, a preferred embodiment of the present invention will be described. 
1. Structure of System 

A system is designed for automatically generating optimum machining condition data, and is composed 
30 of a central processing unit (CPU) 1, a read only memory (ROM) 2, a random access memory (RAM) 3. an 
interface 5, a keyboard 4 and a display device 6 having a cathode ray tube (CRT), as shown in FIG. 1 . The 
system is connected with a numerical controller (not shown) through an interface (also not shown) so that 
machining condition data generated in the system are transferred to the numerical controller. The numerical 
controller controls a not shown grinding machine in accordance with the the machining condition data. 
35 In the ROM 2, there are formed a control program area 21 in which a control program for general 
control is memorized, a neural network area 22 in which a program constituting a neural network 10 is 
memorized, and a learning program area 23 in which a program for learning operation of the neural network 
is memorized. In the RAM 3, there are formed an input data memory area 31 for memorizing input data, a 
teaching data area 32 for memorizing teaching data, discrimination data memory area 33 for memorizing 
ao discrimination data which indicate whether each component of the teaching data is effective component or 
ineffective component, and a connection weight memory area 34 for memorizing connection weights of the 
neural network. The input data in the input data memory area 31 and the teaching data in the teaching data 
memory area 32 constitute a data base for learning and renewal leaning operations for a neuraJ network 10. 

45 2. Neural Network 

The neural network 10 in this embodiment is composed of an input layer LI, an output layer LO and an 
intermediate layer LM, as shown in FIG. 2. 

The input layer LI has plural input nodes whose total number is e, the output layer LO has plural output 
so nodes whose total number is g, and the intermediate layer LM has plural output nodes whose total number 
isf. 

The output O'j of the j-th node of the i-th layer is calculated by using the following equations, wherein i 
is an integer larger than 1 : 

55 O'j = f(fj) (1) 
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iij - E Wi-l kf ij • i"l k + Vij (2) 

5 

f(x) = 1/{1 +exp(-x)} (3) 

where Vj is a bias for the j-tti node of the i-th layer, and W"Vj a connection weight between the k-th 
node of the {i-1)-th layer and the j-th node of the i-th layer. 
io Since the first layer directly outputs values which are input thereto without operation of the above 
expressions, the output value 0 1 j of the j-th node of the input layer (the first layer) U is equal to the value 
input thereto. 

Detailed operation of the neural network 10 having three-layer structure will now be explained with 
reference to the flowchart shown in FIG. 3. 
15 The operation is carried out by executing a program memorized in the neural network area 22 of the 
ROM 2 with reference to connection weights memorized in the connection weight memory area 34 of the 
RAM 3. 

In step 100, the output values O 1 ) of respective nodes of the input layer (the first layer) LI are applied to 
the nodes of the intermediate layer (the second layer) LM, and then the j-the node of the second layer 
20 executes an operation to obtain the sum of products expressed by using the following expression: 



I 2 i = S Wlfc^-j . Qljc + V2j (4) 
* k=l 

Then, in step 102, the output values of the nodes of the intermediate layer (the second layer) LM are 
calculated by using the sigmoid function of the values calculated by using the expression (4). The output of 
30 the i-th node of the second layer is calculated by using the following expression: 

0 2 , = f^j) = 1/{1+expH2|)} (5) 

The output values O 2 ) is applied to the nodes of the output layer (the third layer) LO. 
as Then, in step 104, the sum of products of the inputs applied to each of the nodes of the output layer 
(the third layer) LO is calculated by using the following expression: 

f 

40 l3j « 2 W2 k# 3 j . 0 2 k + v3j (G) 

Then, in step 106, the output values O 3 ] of the nodes of the output layer LO are calculated by using a 
46 sigmoid function similarly to the expression (5). . 

0*j = f(Pj) = 1/{1+exp(-Pj)} (7) 

3. Structures of Input Data and Teaching Data 

50 

A data base used for learning and renewal learning has a structure as shown in FIG. 5. Plural input data 
are indicated by reference signs D1 through Dn while corresponding teaching data are indicated by 
reference signs E1 through En. These n pairs of input data and teaching data are accumulated during an 
initial learning period of the neural network and during the time when the neural network is actually used. 
55 The input data are defined as follows. Each input data comprises e pieces of data or components which 
correspond to e pieces of the input nodes (the nodes of the input layer LI). Further, m-th input data is 
indicated as Dm, and a piece of data (data item) of the input data corresponding to j-th input node is 
indicated as d^. In the other word, D m is a vector, while d^ is a component thereof. 
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Namely. D m is expressed as follows: 

Dr, = (dmi, dmg. , d^. <*me) (8) 

s Further, n sets of input data are expressed as Di. D2 D ni respectively. A group 

comprising n sets of input data is referred to as input data group D in the following explanation. 

When the expression (4) is applied to the input data Dm.component ^ is substituted for 0\ of the 
expression (4). 

Similarly, Et , , En are defined as follows. Each teaching data comprises g pieces of data which 

10 correspond to g pieces of the output nodes (the nodes of the output layer LO). Further, m-th teaching data 
is indicated as Em, and a piece of data (data item) of the teaching data corresponding j-th output node is 
indicated as e^. In the other word, En, is a vector, white e^ is a components thereof. 

Namely, Em is expressed as follows: 

15 Em = (e m1 , e^, emg-i , e mg ) (9) 

Further, n sets of teaching data are expressed as Et. E2 " E^, E„, respectively. A group 

comprising n sets of teaching data is referred to as teaching data group E in the following explanation. 
Date (P, S, R) indicating the kind of data is given to each of the input data and teaching data so that the 

20 input data and teaching data are classified into three data types or categories, namely, type P. type S and 
type R. Data of type P are data for a first grinding cycle for grinding a cylindrical surface only, data of type 
S are data for a second grinding cycle for grinding a cylindrical surface with a shoulder, and data of type R 
are data for a third grinding cycle for grinding a cylindrical surface with a corner. FIG. 7 shows data items 
included in the input data for the above three types of grinding cycles, while FIG. 8 shows data items 

25 included in the teaching data for the above three types of grinding cycles. As is clearly under stood from 
FIGS. 7 and 8, different data items are used for respective grinding cycles. This means that the input data 
and teaching data include ineffective components, whose values have no meaning for the purpose of 
learning operation. 

Therefore, a discrimination data table as shown in FIG. 6 is formed in the discrimination data memory 
30 area 33 of the RAM 3 for memorizing data indicating whether each of the data items is effective component 
or ineffective component. Namely, for each of data items (components) of three types of data, "on" or "off" 
is written. Data items marked by "on" are effective components while data items marked by "off" are 
ineffective components. The effective components are components which are actually used and the values 
of which have meanings. On the contrary, the ineffective components are components which are not used 
as and the values of which do not have meanings. 

4. Learning of Neural Network 

The learning operation of the neural network is carried out during the initial learning period, by 
AO executing the program memorized in the learning program memory area 23 of the ROM 2. The program is 
shown by the flowcharts of FIGS. 4 (a) and 4 (b). The learning of the connection weights, namely, correction 
of the connection weights is carried out by a well-known back-propagation method. 

This learning operation is repeated until respective output data corresponding to plural sets of input 
data become equal to corresponding teaching data. As mentioned above, these input data and teaching 
45 data are memorized in the input data memory area 31 and the teaching data memory area 32. 

In step 200, a variable i representing a data number is set to an initial value 1, and a variable j 
representing the node number of an output node (component number of teaching data) is set to an initial 
value 1. 

In next step 202, i-th input data Dj and i-th teaching data E f are read out from the input data memory 
50 area 31 and the teaching data memory area 32, respectively. 

Then, in step 204, it is judged whether j-th component ey of the i-th teaching data Ej is effective 
components or not by reference to the discrimination data table memorized in the discrimination data 
memory area 33. When the component ejj is effective component the processing moves to step 206 in 
which a learning signal for an node of the output layer LO corresponding to the judged component is 
55 calculated by the following expression: 

Y 3 ] = (e q - O*,) • f(l 3 j) (10) 
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where data number i is omitted from Y 3 ^ O^, and T(x) is the derivative of a sigmoid function. 

Further, the values Pj can be obtained as follows. Firstly, the respective components of input data Dj are 
substituted for 0' k of the expression (4) to obtain the values of Pk for all of the nodes of the intermediate 
layer LM, and then the values of Pk are substituted for the expression (5) so as to obtain output values CP k . 
5 After that, the values d\ are substituted for the expression (6) to obtain the values Pj. The values can 
be obtained by substituting the values for the expression (7). 

On the contrary, when it is judged in step 204 that the component is ineffective component, the 
processing moves to step 208 in which the learning signal is set to zero. 

In next step 210. it is judged whether or not the learning operation has been carried out for all of the 
io output nodes or all of the components. When it is judged that the learning operation has not been carried 
out for all of the output nodes yet the processing moves back to step 204 through step 212 in which the 
variable j is incremented by one. With. this operation, the above learning operation is repeated for a different 
output node. 

When it is judged at step 210 that the learning operation has already been carried out for all of the 
is output nodes, the processing moves to step 214 in which a learning signal for r-th node of the intermediate 
layer LM is calculated by the following expression: 



20 



Y 2 r ■ f'{l2 



g 

» £ Y3 r 



W2 



rt 3 k 



(11) 



The above calculation for learning signals are carried out for all of the nodes of the intermediate layer 

25 LM. 

In step 216, the connection weight of each node of the output layer LO is corrected or modified with a 
correction value. The correction value is obtained by the following expression: 

dWVrft) = P-YyfO 2 ,) + Q-dWVj(t-1) (12) 

where dWVjW represents a variation of the combination weight between the j-th node of the output layer 
LO and the i-th node of the intermediate layer LM in the t-th calculation, dWVjtf-l) is a correction value for 
the connection weight in the (t-1)-th calculation, and P and Q are proportional constants. Therefore, a 
corrected connection weight is calculated by the following expression: 

35 

WVi + dWVj-WV, (13) 

After the processing has moved step 218, the connection weight of each node of the intermediate layer LM 
is corrected. The correction amount of each connection weight is obtained by the following expression: 

40 

dW,,^(t) = P-vyUP,) + Q-dWV^M) (14) 

Therefore, a corrected connection weight is calculated by the following expression: 

45 W 1 ,, 2 ] + dWV 2 j - W 1 ,, 2 ! (15) 

In step 220, it is judged whether or not a series of learning operation using the n sets of input data and 
teaching data has completed. When the series of the learning operation has not completed yet, the 
processing moves to step 222 in which the variable i is incremented by one so that next input data and next 
so teaching data are read from the input data memory area 31 and the teaching data memory area 32, 
respectively, and the variable j is set to the initial value 1 . After that the processing moves back to step 202 
to repeat the learning operation with the next input data and teaching data. 

When it is judged at step 220 that the learning operation using n sets of input data and teaching data 
has been completed, the processing moves to step 224, in which it is judged whether or not the connection 
55 weights have converged. This judgement is carried out by judging whether or not a squared valve of the 
differences between the output data and the teaching data becomes smaller than a predetermined value. 

When it is judged that the connection weights have not converged yet, the processing moves back to 
step 200 for carrying out a second learning operation using the n sets of input data and teaching data. 
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The processing is repeated until K is judged at step 224 that the squared value of the differences 
between the output data and the teaching data becomes smaller than a predetermined value, i.e. until the 
connection weights have converged. 

As described above, when the leaning signal is calculated using any one of ineffective components of 
5 the teaching data, the learning signal is set to zero. Namely, the output error at that time is regarded as 
zero. Accordingly, the connection weight between j-th node of the output layer LO and i-th node of the 
intermediate layer LM is corrected using the previous compensation value of the connection weight as is 
dearly understood from the expression (12). 

Further, since the learning signal Y*j is set to zero, the ineffective component of the teaching data does 
io not affect the learning signals Y 2 ] for the respective nodes of the intermediate layer LM, as is clearly 
understood from the expression (11). Accordingly, ineffective component of the teaching data does not 
affect the connection weight between j-th node of the intermediate layer LM and i-th node of the input layer 
I L, as is clearly understood from the expression (14). 

After ail, the connection weights of the neural network are not affected by the ineffective components of 
75 the teaching data. Accordingly, it is possible to prevent the ineffective components from affecting the result 
of learning which has been carried out using other teaching data in which the components are not 
ineffective but effective. 

5. Renewal of Data 

20 

After the above-described leaning operation, the neural network 10 has a predetermined input-output 
characteristic. 

When the neural network is actually used after such initial learning, new input data and new teaching 
data are input through the keyboard 4, and are added to the data base, and renewal learning is then carried 
25 out Such addition of the input and teaching data and the renewal learning are carried out when undesirable 
output data are output from the neural network 10. 

Before the renewal learning, H is judged whether or not the addition of the new input data and new 
teaching data give an bad influence to the neural network 10. Thus, the new input data and new teaching 
data are added to the data base only if the addition of the new input data and new teaching data does not 
30 give any bad influence to the neural network 10. 

The detail operation will now be explained with reference to FIG. 9. 

In step 300, new input data D n are input to the neural network 10 so that output data O n are calculated 
therein. The output data O n are calculated in accordance with the operation shown in FIG. 3. In next step 
302, the heterogeneousness degree d n of the new teaching data E n is calculated by the following 
35 expression: 

d n = jEn-Onj (16) 

Since the teaching data En and output data O n are vectors whose component number is equal to the 
40 node number g of the output layer LO, d n represents a Euclid distance between both data. 

The output data O n corresponding to the new input data D n represent values which are predicted based 
upon the present input-output characteristic of the neural network 10 which has been determined by the 
initial learning and renewal learning up to the present. Namely, the output data O n are values which are 
determined based upon the tendency of past learning of the neural network 10 up to the present. 
45 Accordingly, the heterogeneousness degree d n indicates how far the teaching data En apart from the 
tendency of past learning. 

In next step 304, the heterogeneousness degree d„ is compared with a threshold value f(x). The 
threshold value f(x) is a function of the number x of renewal learning, as shown in FIG. 10, and the threshold 
value decreases as the number of renewal learning increases. Therefore, as the renewal learning is 

so repeated, the input-output characteristic of the neural network 10 becomes stable and finally converges. 
When the input-output characteristic of the neural network 10 is stable, it is preferred not to drastically 
change the input-output characteristic. On the contrary, when the input-output characteristic of the neural 
network 10 has not converged yet it is preferred to correct the input-output characteristic using the new 
input data and new teaching data which represent a cause-effect relationship in a different condition, 

55 because the learning operation has not completed for sufficient number of possible conditions. 

Before the input-output characteristic of the neural network 10 has not converged yet, the input-output 
characteristic is corrected with the teaching data even when the heterogeneousness degree d n is relatively 
large. After the input-output characteristic of the neural network 10 has already converged, on the contrary, 
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the input-output characteristic is not corrected with the teaching data if the heterogeneousness degree <t, is 
large. 

When it is judged in step 304 that the heterogeneousness degree d„ is equal to or smaller than the 
threshold value f(x), the processing moves to step 306 in which the new input data D„ and new teaching 

s data E„ are memorized in the input data memory area 31 and teaching data memory area 32, respectively. 
After that, the learning operation is carried out using the new data base which comprises the new input data 
D n and new teaching data En, in step 308, so that a new input-output characteristic is obtained. 

On the contrary, when it is judged in step 304 that the heterogeneousness degree d„ is larger than the 
threshold value f(x), the processing moves to step 310 in which there are displayed on the CRT of the 

w display device 6 the heterogeneousness degree d n and a message which informs the operator of the fact 
that the new input data D n and new teaching data E„ are not memorized. 

Obviously, numerous modifications and variations of the present invention are possible in light of the 
above teachings. It is therefore to be understood that within the scope of the appended claims, the present 
invention may be practiced otherwise than as specifically described herein. 

is An apparatus for carrying out learning operation of a neural network which has an input layer, an 
intermediate layer and an output layer. Plural nodes of the input layer are related to plural nodes of the 
intermediate layer with plural connection weights, while the plural nodes of the intermediate layer are also 
related to plural nodes of the output layer with plural connection weights. Although input data composed of 
plural components and teaching-data composed of plural components are used in learning operation, some 

20 of the components are ineffective for the purpose of learning operation. During error calculation between 
output data from the neural network and the teaching data, the apparatus judges whether each of the 
components of the teaching data is effective or ineffective, and output errors corresponding to the 
ineffective components are regarded as zero. The connection weights are thereafter corrected based upon 
the calculated errors. The apparatus further comprises means for adding new input data and teaching data 

25 into a data base, and means for calculating a degree of heterogeneousness of the new teaching data. The 
new input data and teaching data are added to the data base only when the degree of heterogeneousness 
is smaller than a predetermined value. 

Claims 

30 

1. A method of carrying out learning operation of a neural network having at least an input layer 
composed of plural input nodes and an output layer composed of plural output nodes, said input nodes 
and said output nodes being operatively coupled with each other with plural connection weights, said 
method comprising steps of: 

35 reading out from a data base a pair of input data and teaching data, said input data being 

composed of plural components respectively corresponding to said input nodes, and said teaching data 
being composed of plural components respectively corresponding to said output nodes; 

judging whether or not each of the components of said teaching data is ineffective data for the 
purpose of learning operation; 
40 applying said input data to said neural network so that said neural network outputs output data 

which is composed of plural components corresponding to the components of said teaching data; 

calculating differences between the components of said output data and the components of said 
teaching data; 

regarding some differences between some components of said output data and ineffective 
45 components of said teaching data as zero; and 

correcting said connection weights based upon the calculated differences. 

2. An apparatus for carrying out learning operation of a neural network having at least an input layer 
composed of plural input nodes and an output layer composed of plural output nodes, said input nodes 

so and said output nodes being operatively coupled with each other with plural connection weights, said 
apparatus comprising: 

a data base for memorizing plural pairs of input data and teaching data; 

means for reading out from said data base a pair of input data and teaching data, said input data 
being composed of plural components respectively corresponding to said input nodes, and said 
55 teaching data being composed of plural components respectively corresponding to said output nodes; 

means for judging whether or not each of the components of said teaching data is ineffective data 
for the purpose of learning operation; 

means for applying said input data to said neural network so that said neural network outputs 
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output data which is composed of plural components corresponding to the components of said teaching 
data; 

means for calculating differences between the components of said output data and the components 
of said teaching data; 

5 means for regarding some differences between some components of said output data and 

ineffective components of said teaching data as zero; and 

means for correcting said connection weights based upon the calculated differences. 

3b An apparatus for carrying out learning operation of a neural network according to Claim 2, wherein said 
w means for judging comprises a discrimination data table which memorizes data indicating as to whether 
each of the components of said teaching data is effective data or ineffective data, and said judging 
means judges whether or not each of the components of said teaching data is ineffective data by 
reference to said table. 

75 4. An apparatus for carrying out learning operation of a neural network according to Claim 3, wherein said 
neural network further comprises an intermediate layer composed of plural nodes, and said means for 
correcting corrects connection weights between said input layer and said intermediate layer, and 
connection weights between said said intermediate layer and said output layer. 

20 5. An apparatus for carrying out learning operation of a neural network according to Claim 3, wherein said ' 
input data and said teaching data are prepared for each of plural different categories and memorized in 
said data base, and said discrimination data table is composed of plural memory areas for said different 
categories each of which memorizes data indicating as to whether each of the components of said 
teaching data of respective categories is effective data or ineffective data. 

25 

6. An apparatus for carrying out learning operation of a neural network according to Claim 2, said 
apparatus further comprises; 

means for inputting new input data and new teaching data for modifying the input-output char- 
acteristic of said neural network; 
30 means for applying said input data to said neural network so that said neural network outputs 

output data corresponding to said input data; 

means for calculating a degree of heterogeneousness of said new teaching data based on 
differences between said output data and said new teaching data; 

means for judging whether or not the degree of heterogeneousness of said teaching data is smaller 
35 than a predetermine value; and 

means for adding said new input data and new teaching data into said data base only when said 
calculated degree of heterogeneousness is smaller than the predetermined value. 

7. A method of carrying out learning operation of a neural network having at least an input layer 
40 composed of plural input nodes and an output layer composed of plural output nodes, said input nodes 

and said output nodes being operatively coupled with each other with plural connection weights, said 
method comprising steps of: 

reading out from a data base plural pairs of input data and teaching data; 

changing said connection weights through a learning operation with said read input data and 
45 teaching data so that said neural network has an input-output characteristic; 

inputting new input data and new teaching data for modifying the input-output characteristic of said 
neural network; 

applying said input data to said neural network so that said neural network outputs output data 
corresponding to said input data; v 
so calculating a degree of heterogeneousness of said new teaching data based upon differences 

between said output data and said new teaching data; 

judging whether or not the degree of heterogeneousness of said teaching data is smaller than a 
predetermine value; and 

adding said new input data and new teaching data into said data base only when said calculated 
55 degree of heterogeneousness is smaller than the predetermined value. 

& An apparatus for carrying out learning operation of a neural network having at least an input layer 
composed of plural input nodes and an output layer composed of plural output nodes, said input nodes 
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and said output nodes being operatively coupled with each other with plural connection weights, said 
apparatus comprising: 

data base means for memorizing plural pairs of input data and teaching data; 

means for reading out from a data base plural pairs of input data and teaching data; 
5 means for changing said connection weights through a learning operation with said read input data 

and teaching data so that said neural network has an input-output characteristic; 

means for inputting new input data and new teaching data for modifying the input-output char- 
acteristic of said neural network; 

means for applying said input data to said neural network so that said neural network outputs 
w output data corresponding to said input data; 

means for calculating a degree of heterogeneousness of said new teaching data based upon 
differences between said output data and said new teaching data; 

means for judging whether or not the degree of heterogeneousness of said teaching data is smaller 
than a predetermine value; and 
75 means for adding said new input data and new teaching data into said data base only when said 

calculated degree of heterogeneousness is smaller than the predetermined value. 

9. An apparatus for carrying out learning operation of a neural network according to Claim 8, wherein said 
means for judging further comprises means for changing said predetermined value in accordance with 

20 the number of learning operations which have been already accomplished. 

10. An apparatus for carrying out learning operation of a neural network according to Claim 9, said means 
for calculating calculates said degree of heterogeneousness based upon a Euclid distance between 
said output data and said teaching data. 

25 
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